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"Resolution invariant panoramic imaging" 

The invention is described in the following statement: 
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polar image of the scene, and so the pixel density per solid angle increases with the radius of 
the polar image. The unwarping process transforms the image from polar to Cartesian 
coordinates so that the angular coordinate in the original polar image maps to the x-coordinate 
in the unwarped image while the radial coordinate maps to the y-coordinate. Thus the pixel 
5 density in the unwarped image varies from low for small x values which correspond to the 
centre of the original image to high for large x values which correspond to the outer rim of 
the polar image. This is illustrated in Figure 1 which shows the unwarping of an image 
captured with a hyperboloidal mirror. The variation in image quality is clearly evident in the 
unwarped version. 

10 

One way to circumvent this problem is to use a specially designed CCD camera with a polar 
array of pixels with a pixel density which decreases with radius. There are alignment 
problems with such an approach. 

15 In a first aspect this invention provides a panoramic imaging system including an imaging 
device having an image plane and a first field of view, a reflective surface having at least one 
circularly symmetric portion convex in a radial direction disposed in said first field of view 
to provide an expanded panoramic second field of view, the profile of the or each convex 
portion providing a varying gain between the fields of view in the radial direction to limit 

20 variation in the solid angle of view across the image plane of the imaging device. 

Preferably, the profile of the convex portion provides a substantially uniform solid angle of 
view across the image plane. That is, the shape ensures that the resolution in the image is 
invariant to changes in elevation. Thus, where the imaging system involves a device with an 
25 array of uniformly spaced pixels in the image plane, the shape of the reflective surfaces 
results in solid angle pixel density invariance. 

The profile of the reflective surface in polar coordinates is preferably determined by solving 
the equation 

30 
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mounted above a camera to obtain two views of a panoramic scene for stereo disparity range 
finding is known. There are however no proposals concerning specific mirror shapes to 
achieve specific desirable properties. Additionally, known constant gain mirror profiles have 
been generalised to derive a family of such coaxial mirror pair profiles for panoramic stereo 
5 imaging and processing based on disparities in the vertical plane. 

In another aspect this invention provides for range finding using a panoramic imaging system 
containing two resolution invariant mirrors in a coaxial pair. 

10 In a further aspect this invention provides a design for a back to back stereo mirror system 
with the desirable property of equal pixel sharing between two cameras and thus the two 
stereo images. The stereo cone in this case is preferably symmetric in the directions 
orthogonal to the camera axis which is a desirable property for some applications. 

15 In yet a further aspect this invention provides mirrors having minimal intrusive designs, 
which intrude to a minimal extent into the viewing "hemisphere". These are also termed 
forward facing designs. They involve an additional planar mirror and camera relocation 
within the primary reflective surface. The attraction of this arrangement is that the first 
reflective mirror surface profile is the same design as in a more conventional arrangement. 

20 

The invention will be further described, by way of example only with reference to the 
accompanying drawings in which: 

Figure 1 illustrates an unwarping process for a prior art panoramic imaging system; 

Figure 2 schematically shows the relationship between camera image and horizontal 
25 view direction in a panoramic imaging system; 

Figure 3 illustrates geometric relationships between a reflecting surface and a camera 
used to derive mirror profiles according to this invention; 

Figure 4 are graphs showing a comparison of a constant gain mirror with a variable 
gain mirror used in the imaging system according to this invention; 
30 Figure 5 shows ray traced scenes respectively reflected in constant and variable gain 
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between the pixel density and the angle of elevation in the scene or more precisely, the solid 
angle. The mirror gain a, is the relationship between the change in elevation of rays incident 
on the mirror and the change in the angle of rays reflected into the camera as follows 



where d<p is the change in vertical elevation and 30 is the change in angle of reflected rays 
received by the camera. With resolution invariance a becomes a function of image angle 0 
which is related to the radial coordinate in the image, p, as shown in Figure 2. 

10 Consider a mirror profile (r,0) in polar coordinates where r is the radial distance to the 
camera and 0 is the angle from the optical axis of the camera to the point on the mirror 
surface as shown in Fig. 3. The angle of incidence of a light ray relative to the mirror is y 
and the angle of an incoming light ray with respect to the vertical is <J). Then 




(2) 



15 subject to the geometric constraint (from the law of reflection) 



2y + 0 + c|> = 7i 



(3) 



Differentiating (2) and (3) with respect to 0 




/j(p)=UKp 2 

pixels in an area of radius p , where k is the number of pixels per unit area, a constant. 
Differentiating by p gives 



f^ = 2™cp (7) 
dp 



5 Now, the radius in the image, p is related to the radial angle of a ray reflected from the 
mirror, 6 by the focal length of the camera, / (a constant) 

p=/tan(G) (8) 



so differentiating p(p) by 9 and substituting (7) and (8) gives 

10 

dpi?) _ fo(p) dP 



90 dp 86 

= 2KKp/ ^(e) (9) 

= 2tik/ 2 tan(e)(l +tan 2 (6)) 



Now, it is required that the image pixel density be invariant to angle of elevation in the scene 
which leads to more of the scene being imaged towards the perimeter, so 

15 

pip) = p<}> + C(<|>) (10) 
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B 2((j) - <j>) 

' ~ tan 2 (6) - tan 2 (e) (14) 

(j>(6 = 0) = & - tan 2 (9) 

It appears not possible to find an analytical solution to (6) if a is a function of 6, so there is 
no explicit equation for the mirror shape. Instead, a differential equation solver is needed to 
find solutions to (6) over the range of 6 (the mirror surface). 

5 

Fig. 4 shows for comparison a constant gain mirror and a variable gain mirror with the same 
camera field of view and range of elevations imaged. The rays shown are constantly spaced 
in 0, with about 2° between each ray. It is clear from Fig. 4a that in the constant gain case 
these rays are constantly spaced in <}>, with about 8.5° between each ray, and from Fig. 4b, 
10 that the spacing between the rays in the variable gain case increases with increasing <j>. So, 
in the variable gain case, a greater proportion of the scene is imaged towards the outer edge 
of the polar image. This is also shown in Fig. 5, ray traced images reflected in a constant 
gain and variable gain mirror with the same range of elevations visible. 

15 1.2 Panoramic Stereo Using a Variable Gain Mirror 



The mirror arrangement for panoramic stereo with variable gain mirrors will necessarily be 
different than for constant gain mirrors due to the variation of the mirror gain, a. The gain 
must vary in a constant fashion over the entire double mirror so that the constant pixel density 

20 theorem will hold over the entire image. If the minimum and maximum elevations viewed ((j> 
and ^ ) are to be equal for both mirrors in the double mirror system, the range of reflected 
angles (& - 6 ) cannot be equal for the two mirrors. The minimum and maximum angles of 
reflected rays captured by the camera over the entire mirror surface are known from camera 
geometry. Therefore the minimum ray reflected from the lower mirror ((\ ) and the 

25 maximum ray reflected from the upper mirror (0 2 ) are known. So, since (12) holds over the 
entire mirror, B a is constant, and from (14) 
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(17) 



where m n is the gradient of the incident beam to the lower mirror and C n is the equation 
constant. The equation constant is given by 



c u=yx- n 'ii x i 



(18) 



where 



10 



x 2 = r ; sin 0j 
y 2 = r 2 cos 9j 



(19) 



are the Cartesian coordinates of the reflection point (r 1? 0 t ). The gradient of the incident 
beam is found using the law of reflection 



15 



m n - tan 



tan 



+ tan" 



m 



Rl ) 



-tan" 



(20) 



where m Rl is the gradient of the reflected beam from the lower mirror to the camera and 
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Now from (23) and (2), 

£) = Y + — 9 + — tan 2 (0) (25) 



5 so, for the lower variable gain mirror profile 

A-Yx + ^ + ^tan'Ce,) 

similarly for D 2 , for the upper variable gain profile. 

10 So, by substituting (24) into (22) gives the gradient of the variable gain mirror profiles at any 
point. Note that as in the constant gain case, the gradient depends only on 0. 

The equation constants for the incident beam equations from (18) require the polar 
coordinates of the reflection points from each mirror, (r 7 , 0 7 ) and (r 2 , d 2 ) • Since me variable 
15 gain mirror equations are not known exactly, r, and r 2 must be found using a differential 
equation solver to find solutions to (26) at 0, and 0 2 . 

2 Back-to-back Stereo Mirror Families 

20 A key disadvantage of single camera stereo panoramic systems is that since there are two 
images of the "same" scene, the pixels assigned to each image is half that for non stereo 
panoramic imaging and the two images do not share an equal number of pixels in constant 
gain schemes. Actually, the panoramic stereo double mirror method typically causes the view 
of a scene in one radial direction to be compressed into around 1/4 the field of view of the 
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2.1 The Use of Double Mirrors in a Back to Back Design 

Fig. 10 shows a back to back design incorporating double mirrors. Although the figure shows 
constant gain mirrors, the double mirror can also have a variable gain. The advantage to this 
5 system is that the stereo cone from the back to back configuration combines with the stereo 
cones from the double mirror configuration to increase the total area imaged in stereo. In this 
configuration, the fields of view of each double mirror pair need not be aligned as in previous 
examples. For symmetry about the horizon ^ = (j\ , (fa = 4h > 
<j7 3 = (jTj and 4> 4 = <j> 2 . The constraints 



align the three stereo cones. 

It is also possible to increase the total stereo cone further by allowing the mirror pairs to have 
15 different gains. 

3 Forward Looking Mirror Design 

An example of a forward looking mirror design is shown in Fig. 1 1 . For many applications, 
20 it is desirable to have a panoramic camera looking out from, say, a hemisphere, somewhat 
as an eye of a bird, or perhaps two such on either side of a "nose cone". There are 
aerodynamic considerations or other protrusion considerations which motivate such a 
"forward looking" system. This configuration is termed forward looking because the camera 
faces towards the scene. Either a constant or variable gain mirror (double or single) could be 
25 used for the curved mirror in the system. The planar mirror is an annulus or circle such that 
all rays reflected from the curved mirror are reflected into camera o. The dotted lines in Fig. 
11 show where the reflected rays would converge if the planar mirror was removed and the 



<j>3 = n 



*1 

4>, 
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Fig. 12 shows a design that incorporates the ideas of Sections 2 and 3. It consists of two 
forward looking systems back to back, giving a design reminiscent of a eye mounted on a 
stalk, such as a crab's eye. The "stalk" for this system would be hidden from view by the 
lower planar mirror. 

5 

The foregoing describes only some aspects of the present invention and modifications can be 
made without departing from the scope of the invention. 

j 

10 DATED this 15th day of January 1999. 

THE AUSTRALIAN NATIONAL UNIVERSITY 
DA VIES COLLISON CAVE 
1 5 Patent Attorneys for the Applicant 
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Figure 1: An example of the unwarping process. The image here was cap- 
tured with a hyperboloidal mirror. Note the variation in image quality in the 
unwarped version. This will be present for any mirror shape not designed for 
resolution invariance. 
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Figure 3: Relationships used to derive a family of resolution invariant mirror 
profiles. 




Figure 5: Ray traced scenes reflected in a constant gain and resolution in- 
variant mirror. 
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Figure 7: Ray traced comparison of stereo constant gain and resolution in- 
variant mirror systems. 




Figure 9: Back to back panoramic mirror design. 
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Figure 11: A forward looking mirror design. 



